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PREFACE 


The  Human  Sensory  Feedback  (HSF)  for  Telepresence  Project  at  the  Armstrong 
Laboratory  (AL/CFBA)  was  founded  in  1986  in  what  was  then  the  Harry  G.  Armstrong 
Aerospace  Medical  Research  Laboratory.  The  mission  of  the  HSF  team  is  to  study  the 
human  interface  with  telerobotic  systems  as  well  as  sponsor  the  development  of  human 
interface  technology.  Current  work  concentrates  on  force- reflecting  arm  interfaces,  force- 
reflecting  finger  interfaces,  and  tactile  feedback. 

When  the  author  joined  the  HSF  project  in  1991,  there  were  two  tactile  feedback 
arrays  in  the  laboratory  that  had  been  provided  under  a  Phase  I  Small  Business  Innova¬ 
tion  Research  (SBIR)  contract  with  TiNi  Alloy  Co.,  San  Leandro  CA.  The  devices  could 
be  used  to  stimulate  the  fingertips  of  a  user  to  represent  geometric  patterns  (shapes, 
letters,  etc.),  but  the  stimulators  simply  sat  on  the  lab  bench  and  the  sensory  stimulus 
felt  by  the  user  had  nothing  to  do  with  any  movement  of  the  tactile  stimulator.  After 
characterizing  the  behavior  of  the  array  and  beginning  a  series  of  stationary  tactile  pat¬ 
tern  recognition  experiments  (with  Dr  Jan  Weisenberger,  Ohio  State  University),  the 
obvious  next  step  was  to  create  a  system  in  which  the  tactile  feedback  array  acts  as  a 
virtual  reality  or  teleoperator  feedback  device,  allowing  the  user  to  move  the  stimulator 
array  to  receive  haptic  feedback  corresponding  to  that  movement.  The  HAP  tic  TAC- 
tile  (HAPTAC)  Display  does  just  that.  Experimental  work  with  this  system  is  being 
documented  with  conference  papers  and  journal  articles.  This  special  report  presents 
the  source  code  for  the  system. 
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INTRODUCTION 


Humans  immersed  in  teleoperator  or  virtual  reality  systems  rely  on  sensory  feedback  devices  to 
give  them  a  feeling  of  being  “present”  in  the  remote  or  virtual  environment.  Teleoperator  interfaces 
have  concentrated  on  feeding  back  visual,  joint  position,  and  joint  force  information.  Virtual  reality 
interfaces  have  concentrated  on  stimulation  of  the  human  visual  and  auditory  senses. 

In  the  development  of  teleoperation  and  virtual  reality  interfaces,  the  sense  of  touch  provided 
by  mechanoreceptors  near  the  human’s  skin  surface  has  not  received  as  much  attention  as  the 
other  senses.  Tactile  feedback  is  vital  to  achieving  a  full  sense  of  “presence,”  and  to  manipulating 
the  remote  or  virtual  environment  with  dexterity.  Even  though  considerable  research  exists  on 
the  psychophysiology  of  human  touch,  the  development  of  mechanical  aids  to  stimulate  this  sense 
realistically  has  continued  to  be  a  serious  challenge  [12].  Tactile  feedback  requires  the  development 
of  hardware  displays  that  are  lightweight,  portable,  and  suitable  for  mounting  in  gloves  or  other 
moving  strata.  Kaczmarek  and  Shimoga  have  both  conducted  thorough  reviews  on  progress  to  date 
in  developing  tactile  feedback  displays  [8,  13,  14]. 

A  planar  haptic  exploration  system  with  a  mobile  tactile  stimulator  array  has  been  developed  at 
the  Armstrong  Laboratory.  A  multi-pin  tactile  feedback  array  moves  over  a  digitizer  pad,  providing 
stimulation  to  the  user  to  represent  a  virtual  surface.  Virtual  surfaces  have  a  cell  resolution  match¬ 
ing  the  spacing  of  the  stimulator  pins  on  the  feedback  array.  The  two  arrays  used  for  development 
of  the  HAPTAC  system  have  pin  spacings  of  3  mm  and  2.5  mm,  although  a  display  recently  deliv¬ 
ered  to  the  Armstrong  Laboratory  has  pins  spaced  1.4  mm  apart.  Small  design  advances  should 
allow  current  technology  to  achieve  1  mm  pin  spacing.  The  pins  are  actuated  by  shape-memory 
alloy  (SMA)  wires  driven  by  pulse-width  modulated  (PWM)  currents. 

Characterization  of  the  feedback  array  has  been  accomplished  with  a  laser  vibration  sensor  [5]. 
Tactile  pattern  recognition  experiments  with  a  stationary  array  are  mentioned  briefly,  as  they  lay 
the  groundwork  for  investigation  of  the  haptic  system  with  a  mobile  stimulator  array.  The  planar 
haptic  exploration  system  has  only  recently  been  developed,  and  experimental  work  is  in  the  earliest 
stages. 
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PREVIOUS  WORK 


Boyd  et  al.  used  a  Telesensory  Systems  Optacon  II  tactile  display  to  construct  a  moving  haptic 
feedback  device  to  represent  graphical  user  interfaces  (icons,  menus,  windows)  on  a  computer  screen 
[2].  The  Optacon  II  has  a  5-column  x  20-row  vibrotactile  array  measuring  1.27  cm  X  2.92  cm. 
With  the  first  system,  the  user  moved  the  mouse  to  control  the  portion  of  the  screen  displayed 
on  the  Optacon  II.  One  hand  controlled  the  mouse,  and  the  other  used  the  tactile  display.  For 
absolute  position  reference  and  one-handed  use,  the  system  was  improved.  The  Optacon  II  vibrators 
were  placed  on  a  planar  absolute  pointing  device  (similar  to  a  graphics  tablet  puck).  Tactile 
enhancements  included  virtual  buttons  and  tactile  “ribs”  on  the  virtual  surface  to  enhance  the 
user’s  sense  of  position. 

Kuc  also  used  an  Optacon  II  connected  to  four  microswitch  motion  detectors  to  conduct  exper¬ 
iments  on  human  interaction  with  information-bearing  tactile  images  [10].  He  focused  on  tactile 
communication  rather  than  virtual  surface  representation.  The  first  of  three  experiments  used  four 
human  subjects  to  determine  the  maximum  number  of  discriminable  patterns  for  various  element 
sizes  and  densities  (similar  style  of  static  pattern  presentation  to  that  of  Hasser  and  Weisenberger 
[5]).  The  second  experiment  determined  maximum  attainable  data  rates  for  different  array  con¬ 
figurations  using  four  subjects  and  the  patterns  from  the  first  experiment.  The  third  experiment 
exposed  two  subjects  to  a  bidirectional  vibrotactile  communication  system  with  which  they  per¬ 
formed  tasks  such  as  switch  toggling  and  simple  vehicle  control  by  interpreting  statically  presented 
tactile  patterns  on  a  virtual  control  panel  and  reacting  with  small  finger  motions  left,  right,  up, 
and  down.  Finger  motions  were  detected  by  four  microswitches  and  could  be  used  either  to  move 
to  a  different  static  display  or  to  toggle  some  state  in  the  system. 

All  tactile  patterns  presented  in  Kuc’s  experiments  were  static;  that  is,  once  presented  they  did 
not  change  until  a  new  pattern  was  presented.  This  precluded  patterns  scanning  across  a  stationary 
finger  in  the  manner  of  Craig  [4]  and  Hasser  et  al.  [5].  Kuc  does  not  report  any  experiments  where 
finger  motions  directly  affected  stimuli  in  a  manner  required  for  haptic  exploration  (a  limitation 
determined  by  the  choice  of  microswitches  rather  than  a  position  sensor). 

Kontarinis  and  Howe  have  constructed  a  9-pin  SMA-actuated  tactile  feedback  array,  with  stim¬ 
ulators  spaced  2  mm  apart  [9].  They  are  preparing  to  mount  the  device  to  a  pair  of  planar 
two-degree-of-freedom  finger  master  controllers,  with  the  tactile  feedback  arrays  perpendicular  to 
the  plane  of  motion  [6].  The  master-slave  testbed  also  has  an  identical  pair  of  slave  manipulators, 
which  will  enable  the  simulation  of  two-finger  planar  telemanipulation  with  tactile  feedback. 

There  are  few  reports  of  direct  tactile  array  feedback  in  telerobotics.  Browse  and  McDonald 
compared  visual  presentations  of  tactile  data  combined  with  visual  scene  image  to  simple  visual 
scene  image  alone,  for  the  ability  of  subjects  to  predict  a  stable  grasp  on  various  block-style  objects 
with  a  parallel  jaw  gripper  [3].  No  experimentation  with  direct  tactile  feedback  to  the  subject’s 
fingers  was  included.  The  authors  concluded  that  visual  presentation  of  tactile  data  increased  the 
chances  that  a  subject  would  accurately  predict  stable  or  unstable  grasp  for  their  experimental 
setup. 
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Massimino  used  discrete  piezoelectric  buzzers  placed  in  various  locations  on  the  hand  to  improve 
performance  in  a  telerobotic  peg-in-hole  task,  but  did  not  investigate  multi-element  tactile  feedback 
arrays  [11]. 

Bliss  et  al.  constructed  two  tactile  feedback  systems  [1],  A  24-element  airjet  system  fed  tactile 
signals  to  each  of  the  24  phalanges  on  a  human  hand  (excluding  the  thumb).  The  second  system  is 
more  relevant  to  this  discussion;  a  4  x  8  array  of  piezoelectric  bimorphs,  spaced  on  0.1-inch  centers, 
was  used  to  feed  back  tactile  information  from  an  on-off  tactile  sensor  with  similar  resolution.  The 
sensor  was  placed  on  a  pair  of  gripper  tongs,  and  a  master-slave  telerobotic  system  was  used  to 
demonstrate  performance  improvement  with  the  tactile  feedback  array.  The  operator’s  finger  is 
centered  over  the  actuator  array.  With  12  rows  on  0.1-inch  centers,  the  array  is  approximately 
1.2  inches  long;  the  array  extends  beyond  the  distal  phalange  (finger  pad)  of  the  index  finger  and 
partially  onto  the  medial  phalange. 

Bliss  showed  that  tactile  array  feedback  slightly  reduced  task  completion  times  (by  about  7%) 
for  a  complex  latch-removal  task,  but  significantly  increased  the  percentage  of  tasks  completed 
successfully  (17-40%  improvement).  Tactile  feedback  became  even  more  important  when  vision 
was  obscured  to  varying  degrees.  Bliss  also  found  that  the  value  of  tactile  feedback  depended  on 
the  novelty  of  the  task,  suggesting  that  tactile  feedback  is  important  for  exploratory  work  where  the 
tasks  are  highly  variable  or  perhaps  unknown  ahead  of  time.  Bliss’  results  showed  that  when  the 
object  is  fragile,  hard  to  find,  or  requires  accurate  positioning  to  be  picked  up,  the  tactile  feedback 
system  increases  efficiency. 

The  efforts  of  Bliss,  Kontarinis,  and  Kuc  are  most  relevant  to  the  current  work.  Bliss  provides 
clear  motivation  to  further  develop  tactile  feedback  arrays,  leaving  many  variables  open  for  future 
investigation.  Kuc’s  trials  with  various  array  element  sizes  and  densities  may  prove  useful,  though 
the  absence  of  a  position  sensor  made  haptic  exploration  impossible.  The  Optacon  II  also  restricted 
Kuc  to  using  the  single  230  Hz  frequency  of  the  piezoelectric  actuators,  with  essentially  on-off 
actuation.  Bliss  was  similarly  restricted  by  his  piezoelectric  actuators.  The  SMA-actuated  array 
used  in  this  work  responds  to  a  range  of  vibration  frequencies  and  step  amplitudes,  widening  the 
range  of  experimental  variables.  Kontarinis’  device  bears  many  similarities  to  the  present  device; 
however,  Kontarinis  uses  slower,  thicker  SMA  wires  (0.75  mm  versus  0.1  mm)  which  produce  higher 
stimulator  pin  forces  (1.2  N  versus  about  0.2  N)  than  the  present  device.  Kontarinis  uses  relatively 
continuous  power  control  generated  by  a  digital-to-analog  converter,  producing  a  non-vibratory 
stimuli.  The  question  of  adequate  stimulator  pin  force  is  an  open  one.  The  Kontarinis  system  is 
designed  so  that  the  stronger  stimulator  pins  must  support  the  force  of  a  human  finger  in  force- 
reflecting  telemanipulation.  The  present  system  uses  a  protective  touch  plate  (described  below) 
that  supports  the  finger,  limiting  load  force  on  the  stimulator  pins. 
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SYSTEM  DESCRIPTION 


30-Pin  Tactile  Feedback  Array 


A  tactile  stimulator  has  been  constructed  for  research  into  human  perception  of  multielement 
tactile  feedback  [7].  The  tactile  stimulator  array  has  30  elements  arranged  in  a  5  X  6  rectangle,  1.2 
cm  wide  and  1.5  cm  long.  Element  tips  are  evenly  spaced  3  mm  apart  in  both  directions.  Each 
element  is  an  “L”-shaped  cantilever  beam,  with  the  long  leg  of  the  beam  lying  horizontally  under 
a  touch  surface.  One  end  of  the  beam  is  anchored;  on  the  other  end,  the  short  leg  of  the  “L” 
protrudes  through  one  of  30  holes  in  the  touch  surface.  The  human  subject’s  finger,  lying  on  the 
touch  surface,  feels  the  stimulator  tips  as  they  rise  upward  through  the  holes. 

Short,  thin,  SMA  wires  which  contract  when  heated  under  prestress  are  connected  at  one  end 
to  the  long  legs  of  the  stimulator  elements.  One  15  mm  long,  0.1  mm  diameter  wire  actuates  each 
stimulator  element.  The  nickel-titanium  SMA  wire,  anchored  first  on  an  insulator  block  above  the 
base  and  again  at  the  middle  of  the  cantilever  beam,  bends  the  beam  upward  when  heated  by 
electric  current.  PWM  signals  provide  the  current  to  the  actuators.  Both  the  period  and  duty 
cycle  of  these  rectangular  signals  may  be  varied,  where  the  duty  cycle  is  the  percent  of  time  the 
binary  pulse  is  “on.”  Figure  1  shows  dormant  and  actuated  stimulator  elements. 
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ACTUATED  STIMULATOR  ELEMENT 


Figure  1:  Dormant  and  Actuated  Stimulator  Element 


In  addition  to  the  desired  z-axis  displacement  towards  the  fingertip,  motion  at  the  tip  has  a 
translation  component  along  the  y-axis,  as  well  as  a  rotational  component.  This  complex  motion 
makes  displacement  sensing  difficult. 

The  30  stimulator  elements  were  acid-etched  out  of  a  single  flat  baseplate  in  a  staggered  diagonal 
pattern  so  that  the  upward-bent  tips  of  the  diagonally  aligned  elements  form  the  rectangular  array. 
Fifteen  elements  are  anchored  from  either  end  of  the  baseplate  at  one  of  two  insulator  blocks.  On 
each  side,  the  15  elements  alternate  with  three  lengths:  short,  medium,  and  long.  Three  lengths 
were  necessary  so  that  elements  could  reach  to  three  of  the  six  rows  from  each  of  the  two  base 
insulator  blocks. 
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The  SMA  wires  attach  to  all  the  elements  at  the  same  distance  from  the  insulator  blocks, 
regardless  of  element  length.  Geometric  variability  occurs  because  the  longer  elements  extend 
farther  upward  when  actuated.  To  see  this,  look  at  the  actuated  element  in  Figure  1  and  extend 
the  distance  between  points  “a”  and  “b,”  causing  point  ‘‘c”  to  rise  slightly  higher.  The  difference 
in  mass  beyond  the  SMA  attachment  point  “a”  causes  variability  in  the  dynamic  behavior  between 
elements  of  different  lengths. 

Force  exerted  against  the  finger  tip  by  an  individual  stimulator  is  on  the  order  of  0.2  N  (about 
20  grams  at  1  g).  Data  on  the  fully-loaded  displacements  and  stiffnesses  of  the  stimulator  elements 
are  not  yet  available. 


9-Pin  Tactile  Feedback  Array 


In  response  to  experience  with  the  30-pin  tactile  array,  TiNi  Alloy  Co.  developed  a  9-pin  array 
as  an  early  prototype  during  a  Phase  II  Small  Business  Innovation  Research  (SBIR)  contract  with 
the  Armstrong  Laboratory.  The  pins  in  the  array  are  each  powered  by  an  SMA  wire  that  extends 
in  a  “V”  shape  from  the  end  of  the  pin  opposite  the  finger  to  the  mounting  base.  The  left  side  of 
Figure  2  shows  a  view  of  the  underside  of  the  “V”-style  tactile  stimulator  array.  The  right  side  of 
Figure  2  shows  the  mounting  box  sitting  atop  the  modified  digitizer  pad  mouse,  with  part  of  the 
cooling  fan  showing  as  a  yellow,  white,  and  red  disc  in  the  upper  right-hand  corner. 


Figure  2:  A  View  of  the  Underside  of  the  9-Pin  Tactile  Stimulator  Array 


The  9-pin  array  has  a  more  forceful  actuation  than  the  30-pin  array;  according  to  TiNi  Alloy 
Co.,  it  can  produce  a  force  equivalent  to  60  grams,  compared  to  20  grams  for  the  30-pin  cantilevered 
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Figure  3:  The  Haptic  Tactile  (HAPTAC)  System 

Each  cell  in  the  virtual  array  corresponds  to  a  3  mm  X  3  mm  square  on  the  virtual  surface 
(matching  the  resolution  of  the  tactile  feedback  device).  In  the  simplest  implementation,  each  cell 
represents  either  a  raised  feature  on  the  virtual  surface  or  remains  a  part  of  a  featureless  background. 
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Description  of  the  Haptic  Tkctile  Feedback  System 


A  two-dimensional  tactile  haptic  display  has  been  constructed,  with  a  tactile  feedback  array 
attached  to  the  pointer  on  a  Calcomp  Drawing  Board  II  digitizer  board.  The  digitizer  board 
provides  high-resolution,  absolute  position  information.  As  the  user  moves  the  tactile  array  across 
the  digitizer  board,  the  array  acts  like  a  moving  window  on  a  virtual  surface  described  by  a  two- 
dimensional  array  in  the  software  program.  The  digitizer  can  provide  position  but  not  rotation 
information,  so  the  user  must  keep  his  or  her  index  finger  pointing  straight  towards  the  top  of  the 
page.  A  photograph  of  the  system  appears  in  Figure  3. 


design.  It  has  rise  times  closer  to  50  ms,  an  improvement  over  the  170  ms  rise  time  of  the  30-pin 
array.  The  9-pin  array  has  a  pin  spacing  of  2.5  mm. 


If  a  tactile  feedback  pin  passes  over  a  “raised”  cell  on  the  virtual  surface,  it  rises  to  stimulate  the 
user’s  finger;  otherwise,  it  remains  dormant.  Pins  can  also  be  actuated  with  less  upward  movement 
to  represent  a  surface  with  no  raised  edges. 


The  system  software  presents  a  single  virtual  surface  at  a  time;  the  user  feels  this  surface  while 
sliding  the  feedback  array  across  the  digitizer  pad.  Experimental  software  can  present  a  series  of 
these  surfaces  in  random  trials.  Subjects  in  the  random  trials  are  prompted  to  enter  the  number  of 
the  surface  they  feel.  Experimental  variables  include  stimulator  pin  frequency,  PWM  duty  cycle, 
and  a  mask  that  can  disable  any  of  the  stimulator  pins.  The  capability  exists  for  automatic  limiting 
or  measurement  of  the  time  required  to  identify  a  virtual  surface  or  perform  a  task. 
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INSTALLATION 


Components 

•  Personal  computer  (equal  to  or  faster  than  IBM  386) 

•  Digitizer  pad 

•  Modified  digitizer  pad  mouse 

•  Tactile  feedback  array  with  amplifier  box 

•  HAPTAC  software 

Software 

Load  all  of  the  software  into  a  dedicated  subdirectory  (e.g.  C:\HAPTAC),  and  create  a  subdi¬ 
rectory  HAPDAT  (e.g.  C:\HAPTAC\HAPDAT)  to  store  any  experimental  data  that  are  created. 
Necessary  software  includes  HAPTAC.EXE,  CCMOUSE.COM,  and  any  .SUR  (virtual  surface) 
files.  The  .SUR  files  use  simple  ASCII  text  (O’s,  I’s,  and  2’s)  to  describe  virtual  surfaces.  Refer  to 
Appendix  A  for  an  example  .SUR  file.  Other  relevant  files  include  the  set  files  (*.SET)  containing 
sets  of  surfaces,  and  mask  files  (*.MSK),  allowing  pins  to  be  selectively  disabled. 

Digitizer  Pad 

•  Connect  the  digitizer  pad’s  serial  cable  to  COMl  on  the  computer. 

•  Connect  the  modified  digitizer  pad  mouse  (flat  acrylic,  with  locator  coil). 

•  Connect  the  1/8”  power  plug  from  the  wall  transformer  to  the  appropriate  socket  (either  on 
the  back  of  the  digitizer  pad  or  on  the  serial  connector,  depending  on  the  model). 

•  Turn  the  digitizer  pad  “on”  using  the  switch  on  the  back  edge. 

Tactile  Stimulator 

•  Plug  the  power  supply  into  the  wall. 

•  If  the  power  supply  is  separate  from  the  control  box  (depends  on  prototype  version),  connect 
them. 

•  Plug  data  cable  into  parallel  port  (PTRl)  on  computer. 

•  Connect  tactile  stimulator  array  to  control  box. 

•  Attach  tactile  stimulator  array  to  modified  mouse  using  Velcro. 
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OPERATION 


Upon  starting  HAPTAC.EXE,  the  user  will  encounter  a  window  offering  five  possible  options: 

Run  experiment  Multiple  surfaces  will  be  presented  to  the  user,  one  at  a  time  and  randomly 
selected.  The  surfaces  will  be  taken  from  a  set  of  up  to  10  surfaces  defined  by  the  user.  Visual 
images  of  all  possible  surfaces  will  be  displayed  on  the  computer  screen  with  a  number  for 
each  surface,  and  the  user  will  be  prompted  to  enter  (using  the  keyboard)  the  number  of  the 
surface  he  or  she  feels. 

Demo  one  surface  A  single  virtual  surface  will  be  presented  to  the  user.  In  a  later  window,  the 
user  will  select  from  a  list  of  the  .SUR  files  contained  in  the  current  directory. 

Set  creation  This  option  allows  the  user  to  assemble  a  set  of  up  to  10  already  existing  .SUR  files 
and  to  give  that  set  a  name  for  later  use  in  experimental  trials. 

EXIT  Exit  the  program  and  return  to  DOS. 

Subsequent  windows  will  prompt  the  user  for  information  (e.g.  the  surface  to  be  demonstrated, 
or  the  subject’s  initials).  One  of  these  windows  will  ask  the  user  to  enter  two  sets  of  parameters 
for  pulse-width  modulation  (PWM)  period  and  duty  cycle.  The  second  PWM  period  must  be  a 
multiple  of  the  first;  this  requirement  results  from  simplifying  assumptions  made  in  the  software. 

Another  window  also  provides  the  user  with  the  ability  to  mask  out  certain  pins.  This  allows 
experiments  to  be  run  with  a  smaller  array  of  active  pins,  or  with  a  particular  pattern  of  pins 
disabled.  Mask  files  are  particular  to  the  display  being  used  (mask  files  for  the  nine-pin  display 
won’t  work  with  the  30-pin  display,  and  vice-versa).  All  mask  files  end  in  .MSK,  and  can  be  edited 
using  a  text  editor. 


Troubleshooting 

•  Mouse  coordinates  in  the  upper  left-hand  corner  of  the  screen  should  change  as  the  mouse 
moves.  Lack  of  coordinates  or  frozen  coordinates  is  an  indication  that  HAPTAC  is  not  com¬ 
municating  with  the  digitizer  pad.  Check  connections,  digitizer  pad  power,  etc. 

•  Problems  can  often  be  solved  by  turning  the  digitizer  pad  “off’  then  ‘‘on,”  and/or  restarting 
the  HAPTAC  program. 

•  If  the  logic  input  to  the  tactile  stimulator  array  seems  to  be  working,  but  the  pins  do  not  rise, 
make  sure  the  “dead  man’s  switch”  is  activated. 

•  Due  to  a  programming  bug,  an  experimental  subject  may  need  to  move  the  mouse  while 
hitting  the  keyboard  key  representing  his  or  her  desired  response  (if  the  mouse  is  not  moving, 
the  program  may  not  check  keyboard  for  input). 
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EXPERIMENTAL  ISSUES 


Since  the  tactile  stimulator  pins  rise  and  begin  vibrating  when  activated,  the  stimulus  resembles 
a  vibratory  motion  superimposed  upon  a  step  displacement.  This  type  of  stimulus  does  not  directly 
correspond  to  earlier  tactile  perception  research  and  devices;  comparison  to,  and  use  of,  previous 
results  must  be  approached  cautiously.  Furthermore,  the  PWM  control  that  causes  this  unique 
behavior  is  not  the  only  successful  way  to  drive  SMA  actuators  for  tactile  feedback  [9]. 

The  development  of  this  system  leads  to  many  questions.  How  will  the  vibratory  behavior  and 
the  speed  with  which  the  pins  are  able  to  rise  through  the  holes  in  the  touch  plate  affect  perception? 
At  what  rate  will  the  subject  be  able  to  move  his  or  her  finger  before  there  is  a  perceptible  lag  in 
the  response  of  the  stimulator  array?  What  perceptual  artifacts  will  be  produced  by  moving  too 
fast?  The  maximum  finger  speed  with  the  present  unoptimized  system  appears  to  be  only  a  few 
centimeters  per  second.  At  excessively  fast  rates  of  motion  the  user  seems  to  feel  nothing,  as  the 
pins  do  not  have  time  to  rise  before  the  stimulator  array  is  positioned  over  different  cells  on  the 
virtual  surface. 

The  fact  that  the  SMA  wires  are  thermally  actuated  and  produce  waste  heat  may  mean  that 
surface  feature  density  (likelihood  that  many  pins  will  be  actuated  repeatedly)  will  affect  stimu¬ 
lator  behavior.  The  effect  may  be  advantageous,  decreasing  pin  rise  time  by  increasing  nominal 
temperature,  or  disadvantageous,  saturating  actuators  with  unusually  high  ambient  temperatures. 

How  fast  should  the  pins  vibrate,  and  how  high  should  they  rise,  in  order  to  produce  effective 
percepts?  How  should  we  define  the  term  “effective?”  Vibration  frequency  can  easily  be  confirmed, 
but  data  on  amplitude  behavior  of  loaded  pins  (finger  placed  on  top  of  stimulator  pins)  will  likely 
be  more  difficult  to  obtain. 

In  addition  to  the  questions  relating  to  individual  pin  control,  questions  exist  concerning  the 
array  as  a  whole.  How  do  pin  density  and  the  number  of  pins  in  the  array  affect  perception?  Each 
of  these  factors  could  be  addressed  in  a  preliminary  way  by  activating  a  reduced  number  of  pins  in 
the  display. 

The  perception  of  touch  with  the  SMA  tactile  array  is  complicated  by  the  fact  that  this  stim¬ 
ulator  delivers  sensory  cues  in  an  alien  fashion,  actuating  some  mechanoreceptors  in  ways  that  do 
not  correspond  to  “real  world”  contacts.  Learning  how  to  use  this  stimulator  with  some  normal 
cues  absent  and  some  alien  cues  present  will  be  a  challenge  for  experimenters. 


System  Sample  Time  and  Latency 


In  the  current  configuration  the  computer  must  turn  the  pins  on  and  off  every  time  they  vibrate 
and  must  also  read  the  digitizer  position.  Since  this  is  done  without  interrupts,  the  maximum 
rate  at  which  the  program  can  sample  the  digitizer  pad  location  is  determined  by  the  period  of 
the  lowest  frequency  stimulus  being  used  (the  longest  period  in  use).  Only  after  the  program  has 
completed  a  complete  on-off  cycle  will  it  check  the  digitizer  position.  Thus,  for  a  10  Hz  stimulus 
(close  to  a  worst-case  scenario),  the  digitizer  pad  position  would  only  be  sampled  at  10  Hz  (every 
100. ms).  The  digitizer  sampling  rate  might  be  improved  by  moving  to  an  interrupt-driven  scheme. 
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Ideally,  the  sample  period  would  be  much  less  than  50  ms.  Exactly  how  low  a  sample  period  would 
be  useful  depends  on  the  ability  of  the  SMA  wires  to  respond  to  rapidly- changing  commands.  At 
some  point,  the  SMA  actuator  bandwidth  rather  than  the  computer  control  rate  will  be  the  limiting 
factor.  Indeed,  this  is  probably  the  case  with  high  frequency  operation,  where  the  computer  checks 
the  digitizer  pad  at  200  Hz  and  the  magnitude  of  vibration  of  the  pins  begins  to  drop  off. 

System  latency  must  also  be  considered.  It  takes  a  computer  running  a  serial  port  at  9600  baud 
about  6  ms  to  read  the  six-byte  packet  of  information  from  the  digitizer  pad.  It  also  takes  time 
for  the  nine  or  30  bits  of  information  to  be  fed  into  the  register  in  the  SMA  driving  hardware,  but 
this  delay  is  dwarfed  by  that  of  the  digitizer  pad.  No  matter  how  fast  the  control  loop  runs,  the 
control  algorithm  will  always  be  using  somewhat  old  information.  At  some  point  this  may  become 
a  system  limitation  (again,  the  bandwidth  limitation  of  the  SMA  actuators  may  keep  this  delay 
from  becoming  apparent  to  the  user). 
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OBJECT  LD.  WITH  MOVING  VERSUS  STATIONARY 

FINGER 


The  ability  of  subjects  to  discriminate  the  simple  patterns  of  Figure  4  has  been  tested  with 
subjects’  fingers  resting  upon  a  stationary  feedback  array  [5].  Subjects  were  able  to  identify  the 
patterns  with  accuracy  approaching  100%.  These  data  provide  a  baseline  against  which  to  test  the 
mobile  stimulator  array.  Subjects  were  also  tested  for  their  ability  to  discriminate  between  wave¬ 
like  line  stimuli  moving  in  eight  different  directions  (up,  down,  left,  right,  and  along  both  directions 
on  the  main  diagonals)  [5].  These  direction-of-motion  tests  also  yielded  near  100%  accuracy.  In 
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Figure  4:  Static  Pattern  Icons  Used  in  Perception  Experiments 

experiments  currently  underway,  the  patterns  will  be  scrolled  from  right  to  left  across  the  subject’s 
finger  as  it  rests  passively  on  a  stationary  feedback  array.  In  a  later  experiment,  the  patterns  will 
be  presented  one  at  a  time  on  the  virtual  surface,  in  random  trials.  The  subjects  will  be  free  to 
move  the  stimulator  in  two  dimensions  to  identify  the  pattern  on  the  virtual  surface.  Preliminary 
experimental  results  indicate  that  identification  of  single-line  patterns  (numbers  0,  1,  6,  and  7)  will 
be  extremely  easy,  while  identification  of  double-line  patterns  (numbers  2,  3,  4,  and  5)  will  require 
more  training. 
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BASIC  OBJECT  REPRESENTATION  ISSUES 


Normal  objects  felt  during  a  task  do  not  resemble  a  grid  of  vibrating  pins.  Very  basic  choices 
must  be  made  for  object  representation,  such  as  whether  to  represent  the  shapes  on  the  virtual 
plane  by  solid  filled-in  shapes  or  hollow  edge-only  shapes.  Pins  positioned  over  the  center  of  a 
filled-in  virtual  rectangle  would  vibrate,  while  pins  over  the  center  of  a  hollow  rectangle  would  not. 

Earlier  experiments  with  vibrotactile  arrays  such  as  the  Optacon  have  been  limited  to  a  pre¬ 
determined  stimulus  amplitude  and  frequency.  Pins  representing  the  edge  of  a  shape  and  pins 
representing  the  centers  of  filled-in  shapes  behaved  similarly.  People  clearly  do  not  perceive  edges 
of  raised  shapes  in  the  same  way  they  perceive  the  textured  surfaces  of  the  shapes.  This  will  influ¬ 
ence  more  than  the  outcome  of  the  filled  in  versus  hollow  question.  If  people  perceive  edges  and 
textures  differently,  should  the  two  be  represented  with  different  stimulus  parameters? 


Variable  Duty  Cycle  with  Motion 


When  a  finger  is  stationary  on  a  non- vibrating  object,  most  mechanoreceptors  are  not  stimu¬ 
lated,  or  have  adapted  to  the  stimulus,  depending  on  the  degree  of  indentation  of  the  skin  surface. 
The  situation  in  which  mechanoreceptors  are  not  excited  by  a  given  stimulus  is  referred  to  as  “stim¬ 
ulus  failure”;  this  is  a  misleading  term,  since  the  stimulus  is  simply  outside  the  sensing  envelope  of 
the  mechanoreceptor  and  no  pathological  failure  has  occurred. 

The  current  system  vibrates  the  pins  whenever  the  array  is  positioned  over  a  virtual  feature. 
This  constant  stimulation  may  limit  the  realism  of  the  percept,  countering  “stimulus  failure”  or 
adaptation  that  would  be  present  when  sensing  non- vibratory  stimuli.  A  method  to  mimic  “stimulus 
failure”  or  adaptation  by  reducing  the  magnitude  of  the  stimulus  might  be  useful.  It  is  relatively 
easy  to  detect  a  change  in  the  x-y  position  of  the  mouse  and  activate  the  stimulator  pins  only  when 
the  subject  moves  his  or  her  finger.  Instead  of  turning  off  the  pins  when  the  subject  remains  still, 
the  duty  cycle  could  be  dropped  by  5  or  10%  to  keep  the  wires  heated,  producing  a  more  subtle 
reduction  of  stimulus  intensity. 

This  technique  was  implemented  so  that  as  soon  as  the  finger  stops  moving,  the  stimulation 
stops.  This  percept  was  noticeably  different.  The  first  subject  (BD)  did  not  notice  a  great  difference, 
but  he  may  have  been  responding  to  the  question,  “Is  there  an  object  on  the  virtual  surface  when 
you  hold  your  finger  still?”  rather  than  to,  “Do  you  actually  feel  stimulation  when  you  hold 
your  finger  still?”  BD  had  been  using  the  stimulator  for  quite  a  while;  he  stated  that  the  case 
of  constant  vibration  (even  when  finger  is  still)  did  not  feel  as  pronounced  when  his  finger  was 
still.  This  perception  could  have  been  caused  by  adaptation,  or  by  ambient-temperature  induced 
bandwidth  loss  in  the  SMA  wire,  or  by  some  unknown  factor.  Other  subjects,  PW  and  DN,  felt 
that  the  sudden  removal  of  stimulus  when  they  stopped  moving  their  finger  was  distracting  and  did 
not  seem  to  mimic  a  real  experience  adequately.  Less  drastic  reductions  in  duty  cycle  may  produce 
a  more  subtle  effect  closer  to  adaptation.  Further  experimentation  will  be  required  to  determine  if 
this  approach  adds  any  value  to  the  percepts. 
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FUTURE  WORK 


Recently  Completed  or  in  Progress 


Early  work  published  by  this  author  and  J.  W.  Weisenberger  studied  tactile  pattern  perception 
using  the  SMA  feedback  arrays,  but  did  not  include  the  ability  for  the  user  to  haptically  explore  a 
virtual  surface  [5].  Research  has  proceeded  more  or  less  continuously  since  that  early  work  and  since 
the  establishment  of  HAPTAC.  Immediately  after  HAPTAC  became  operational,  Weisenberger  and 
this  author  studied  tactile  object  perception  on  HAPTAC  using  the  simple  patterns  developed  for 
[5].  That  effort  also  involved  more  complicated  letter  patterns  perceived  in  static,  scanned,  and 
haptic  modes  [15].  In  subsequent  work,  Weisenberger  et  al.  studied  the  effect  of  varying  the  field 
of  view  (number  of  pins  at  a  given  spacing)  on  pattern  perception  with  HAPTAC  [16].  The  most 
recent  work  to  be  published  by  Weisenberger  et  al.  varies  presentation  time  of  scanned  and  statically 
presented  patterns,  attempting  to  control  for  some  of  the  advantages  caused  by  the  subject’s  ability 
to  repeatedly  explore  a  pattern  in  the  haptic  mode  [17]. 


Miscellaneous  Thoughts 


One  future  task  might  be  to  explore  a  “socket”  on  a  tactile  field,  then  to  identify  the  correspond¬ 
ing  “plug”  on  a  video  screen.  Subjects  might  be  asked  to  match  triangular,  square,  or  round  “pegs” 
(positive  images)  and  “holes”  (negative  images).  Subjects  could  be  asked  to  count  the  number  of 
like  shapes  on  the  virtual  surface,  with  or  without  interference  from  other  shapes. 

The  ability  to  represent  edges  so  that  subjects  can  follow  them  with  their  fingers  may  turn  out 
to  be  a  significant  strength  of  this  system,  since  it  can  represent  line  stimuli  so  well.  Subjects  might 
be  tasked  to  follow  a  complex  edge  with  distracting  edges  in  a  confusing  environment.  They  might 
try  locating  a  virtual  “D”  connector  socket  on  the  back  of  an  imaginary  panel.  For  both  shape 
detection  and  edge  following  tasks,  subjects  would  be  expected  to  show  improvement  in  completion 
times  with  an  increase  in  the  number  and  density  of  stimulator  pins. 


TacGraph 


A  graphics  display  program  for  blind  individuals  has  been  developed  by  joining  the  HAPTAC 
system  to  the  shareware  graphics  driver  ’’gnuplot.”  The  interface  is  essentially  a  special  gnuplot 
device  driver,  with  output  being  to  the  virtual  surface  rather  than  a  printer  or  the  screen.  A  copy  of 
the  virtual  surface  is  also  displayed  on  the  screen  for  the  sake  of  sighted  users.  Braille  titles  and  axis 
labels  appear  on  the  virtual  plots.  Multiple  frequencies  of  pin  vibration  can  be  used  to  represent 
different  data  curves.  Voice  synthesis  using  a  Soundblaster  board  gives  the  user  information  on 
demand  about  plot  title,  axis  labels,  cursor  position,  magnitude  ranges,  and  axis  ranges.  A  patent 
application  has  been  made,  and  an  Armstrong  Laboratory  technical  report  will  likely  follow. 
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APPENDIX  A:  EXAMPLE  SURFACE  FILE 


SQUARES. SUR  10 
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Figure  5:  Example  Surface  File:  I’s  and  2’s  Denote  Features,  O’s  Denote  Background 
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APPENDIX  B:  DIGITIZER  PAD  SETTINGS 


The  digitizer  has  two  banks  of  Boolean  flags  that  can  be  set  using  the  row  of  buttons  at  the  top  of 
the  digitizer  pad.  The  settings  of  these  flags  determine  the  operating  behavior  of  the  digitizer  pad. 
The  Calcomp  manual  has  some  useful  information,  but  was  not  designed  for  system  developers,  so 
this  appendix  records  some  “lessons  learned.” 

Access  the  flag  banks  by  moving  the  mouse  pointer  (the  localizer  coil)  to  the  upper  left-hand 
corner,  over  the  button  labeled  “CONFIG/EXIT.”  With  the  mouse  pointer  positioned  over  the 
“CONFIG/EXIT”  button,  press  the  “0”  button  on  the  mouse  housing  (note  that  with  the  current 
version  of  HAPTAC,  the  mouse  has  been  disassembled  so  that  the  coil  and  housing  are  separate). 
Move  the  pointer  coil  over  the  “A”  button  and  notice  that  the  “POWER”  pilot  light  is  on.  This 
light  will  be  used  to  indicate  the  states  of  various  flags.  Move  over  the  “B”  button  and  notice  that 
the  “POWER”  light  goes  off.  This  means  that  you  are  currently  working  with  Bank  “A.”  Pressing 
the  “0”  button  on  the  mouse  housing  while  over  the  “B”  button  will  switch  to  Bank  “B.”  Move 
the  coil  over  the  numbered  buttons  1-18,  and  watch  the  “POWER”  light  to  see  which  flags  are  set 
“high.”  Pressing  the  “0”  button  on  the  mouse  housing  while  over  a  numbered  button  will  toggle 
the  corresponding  flag  for  whichever  bank  is  currently  activated  (A  or  B).  To  exit  the  configuration 
mode  and  return  to  digitizer  pad  operation,  toggle  the  “CONFIG/EXIT”  button. 

Considerable  trial  and  error  went  into  determining  the  proper  settings.  The  Calcomp  user’s 
manual  was  moderately  helpful,  as  was  Calcomp  technical  support.  The  Human  Sensory  Feedback 
group  owns  two  versions  of  the  digitizer  pad:  Digitizer  Pad  II  and  Digitizer  Pad  III.  The  manual 
for  Digitizer  Pad  II  may  actually  contain  more  detailed  formatting  information.  Most  recently,  the 
settings  that  have  been  used  are: 

Bank:  Flags  that  should  be  set  high: 

A:  1,  2,  4,  6,  9,  10,  12,  14,  15,  16,  18 

B:  3,  4,  10,  14,  17,  18 

Unfortunately,  these  settings  do  not  always  work.  There  is  a  program,  CCMOUSE.COM,  that 
was  shipped  with  the  digitizer  pad.  These  are  the  default  settings  used  by  that  program.  In  a  recent 
experiment,  manually  setting  the  flags  failed,  but  running  and  then  exiting  CCMOUSE.COM  set  the 
flags  to  the  states  bsted  above,  and  the  digitizer  pad  worked  as  expected.  Although  manual  setting 
does  seem  to  set  flag  states  (as  indicated  by  the  checking  with  the  mouse  and  “POWER”  light), 
seemingly  identical  state  settings  had  different  behaviors,  depending  on  whether  they  were  manually 
loaded  or  set  using  CCMOUSE.COM.  This  situation  arose  while  this  author  was  programming 
HAPTAC  to  use  the  “proximity”  indicating  option  of  the  digitizer.  This  option  indicates  whether 
the  pointer  coil  is  on  or  oflT  the  digitizer  pad. 
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APPENDIX  C:  LIST  OF  SUPPLIERS 


BORLAND  INTERNATIONAL 
P.O.  BOX  660001 
1800  GREEN  HILLS  ROAD 
SCOTTS  VALLEY  CA  95066-0001 
PHN:  408-438-5300 
supplied  C  compiler 


CALCOMP 

DIGITIZER  DIVISION 
14555  N  82ND  STREET 
SCOTTSDALE  AZ  85260 
PHN:  800-458-5888 
FAX:  602-948-5508 


TINI  ALLOY  COMPANY 
1621  NEPTUNE  DR 
SAN  LEANDRO  CA  94577 
PHN:  510-483-9676 
FAX:  510-483-1309 
Email:  tinil@holonet.net 
Email:  tini@aol.com 
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